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1 .  SCOPE 


1.1  SUMMARY 


a.  The  procedures  outlined  in  this  TOP  provide  general  guidance  for 
determining  the  intelligibility  characteristics  of  voice  communication  equip¬ 
ment.  The  cumulative  test  results  will  allow  an  estimate  to  be  made  of  the 
effectiveness  of  the  test  item  performance  in  the  environment  of  intended  use. 

b.  The  Specific  tests  to  be  performed,  along  with  their  intended  objec¬ 
tives,  are  as  follows: 

(1)  Harvard  Phonetically  Balanced  (PB)  Word  List  Testing  -  The 
objective  of  this  subtest  is  to  obtain  a  percentage  numerical  score,  referred 
to  as  articulation  score  (AS),  for  the  test  communications  link  effectiveness. 

(2)  Automatic  Voice  Intelligibility  Analysis  System  (VIAS)  Testing  - 
The  objective  of  this  subtest  is  to  obtain  a  numerical  score  of  the  test  com¬ 
munications  link  effectiveness,  referred  to  as  the  articulation  index  (AI). 

The  use  of  this  automated  equipment  provides  high  speed,  low  cost  data  analy¬ 
sis  to  meet  the  needs  of  test  requirements. 

(3)  Appendix  C  provides  a  separate  nonengineering  Subjective  Voice 
Transmission  Quality  Test  procedure.  The  procedure  is  based  on  rating  ele¬ 
ments  from  ITU  Radio  Regulations,  Geneva,  1959,  but  other  authoritative  rating 
elements  can  be  used,  i.e.,  Communications  Instructions  Radiotelephone 
Procedure,  ACP  125(D)  with  changes  1  and  2,  July  1970. 

(4)  The  diagnostic  rhyme  test  (DRT)  and  other  word  lists  do  not 
exercise  the  full  spectrum  phonemes  as  does  the  Harvard  Phonetically  Balanced 
Word  List.  This  TOP  uses  only  the  above  listed  specific  tests. 

*This  TOP  supersedes  MTP  6-2-521,  6  August  1970. 
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1.2  LIMITATIONS 


a.  The  AS  is  a  standard  measure  of  intelligibility  based  on  PB  word 
lists.  This  method  produces  a  good  comparative  measure  but  does  not  have  a 
simple  translation  function  for  determining  acceptability  for  different  field 
applications,  which  are  affected  by  such  factors  as  message  content  and  struc¬ 
ture,  user  experience,  and  familiarity  with  material,  as  well  as  the  purpose 
and  circumstances  of  communication. 

b.  The  AS  does  not  allow  a  detailed  analysis  of  where  intelligibility 
loss  occurs  such  as  that  which  may  be  provided  by  the  diagnostic  rhyme  test 
(DRT)  or  other  phoneme  discrimination  or  confusion  measurements. 

c.  While  providing  a  fast  and  economical  measure  of  intelligibility,  the 
VIAS  has  a  limited  capability  to  evaluate  the  effects  of  speech  processing. 

It  must  be  validated  and  calibrated  relative  to  a  more  complete  measure  such 
as  AS  before  being  used  to  predict  performance  of  systems  employing  new  mod¬ 
ulation  or  voice  coding  techniques. 

d.  The  engineering  test  (body  of  this  TOP)  and  the  subjective  test  (app 
C)  are  separate  and  distinct  test  procedures  and  are  not  intended  to  comply, 
refute,  or  conflict  with  other  methods;  i.e.,  HFE  TOP  1-2-610,  the  linear 
predictive  coding  technique  (LPC)  or  other  authoritative  sources. 

2.  FACILITIES  AND  INSTRUMENTATION 

a.  Voice  Scoring  Facility. 

b.  Voice  Intelligibility  Analysis  System  (VIAS). 

c.  Test  apparatus,  as  required  for  operational  tests  of  the  test  item. 

d.  Support  equipment,  as  required  to  operate  test  item. 

3.  preparation  FOR  TESTING 

Upon  establishing  the  scheduled  availability  of  the  test  item,  the  test 
officer  and  engineer  shall  ensure  availability  of  appropriate  facilities  and 
coordinate  the  availability  of  test  support  requirements.  This  support  will 
include  personnel,  equipment,  maintenance,  spare  parts,  and  instrumentation. 
Special  attention  shall  be  given  to  timely  provision  of  additional  supplies  or 
equipment  not  readily  available  to  the  test  site. 

3.1.  Preparation  for  Test 

Data  to  be  recorded  prior  to  testing  shall  include,  but  not  be  limited 
to: 

a.  Nomenclature,  serial  number(s),  manufacturer's  name,  and  function  of 
the  item(s)  under  test. 
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b.  Nomenclature,  serial  number,  accuracy  tolerance,  caliratlon 
requirements,  and  last  calibration  date  of  test  equipment  selected  for  the 
tests . 

c.  Damages  to  the  test  item(s)  Incurred  during  transit  and/or  manufac¬ 
turing  defects. 

4.  TEST  CONTROLS 


4.1  Test  Item 

a.  Record  the  following  information:  nomenclature,  serial  number(s), 
manufacturer's  name,  and  function  of  the  item(s)  under  test. 

b.  Ensure  that  all  test  personnel  are  familiar  with  the  required 
technical  and  operational  characteristics  of  the  item  under  test. 

c.  Review  all  instructional  material  issued  with  the  test  item  by  the 
manufacturer,  contractor,  or  government,  as  well  as  reports  of  previous 
similar  tests  conducted  on  the  same  types  of  equipment.  Keep  these  documents 
readily  available  for  reference. 

d.  Prepare  record  forms  for  systematic  entry  of  data  including  the  pre¬ 
test  equipment  record,  chronology  of  test,  test  results,  and  such  observations 
and  measurements  that  would  be  of  value  in  analysis  and  final  evaluation. 

e.  Prepare  adequate  safety  precautions  to  provide  safety  for  personnel 
and  equipment.  Ensure  that  all  safety  standing  operating  procedures  are  ob¬ 
served  throughout  the  test. 

4.2  Test  Equipment 

a.  Select  test  equipment  that  is  in  keeping  with  the  state-of-the-art 
and  with  calibration  traceable  to  the  National  Bureau  of  Standards. 

b.  Record  the  following  information:  nomenclature,  serial  number,  ac¬ 
curacy  tolerances,  calibration  requirements,  and  last  calibration  date  of  test 
equipment  selected  for  the  test. 

c.  Check  the  magnetic  tape  equipment  for  operational  readiness  and  make 
all  adjustments  required  by  the  operational  manuals. 

d.  Load  the  master  tapes  on  the  tape  playback  equipment  and  check  for 
proper  operation. 

e.  Calibrate  the  VIAS  equipment  in  the  manner  prescribed  in  the  oper¬ 
ational  and  technical  manuals. 
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5 .  TEST  CONDUCT 


5.1  PB  Word  List  Testing 

a.  Set  up  the  voice  communication  system  or  equipment  in  one  of  the  two 
basic  testing  configurations,  closed-link  or  open-field.  All  testing  con¬ 
figurations  should  have  two  high-fidelity  magnetic  tape  recorders  —  a  play¬ 
back  recorder  at  the  transmitting  site  and  a  receiver  recorder  at  the  re¬ 
ceiving  site. 

b.  Transmit  a  PB  word  list  on  the  playback  recorder  through  the  com¬ 
munications  test  link  to  the  receiver  recorder  at  the  receiving  site. 

c.  Repeat  the  recording  procedures  under  the  conditions  stated  by  the 
test  plan.  The  exact  conditions  must  be  recorded  so  that  follow-on  tests  can 
duplicate  the  same  conditions  to  the  maximum  extent  possible. 

d.  Score  the  PB  word  list  test  in  the  Voice  Scoring  facility  (see  app 
A). 

5.2  Automatic  Testing 


f 


a.  Connect  the  test  signal  generator  and  VIAS  to  the  test  link  as 
indicated  in  appendix  B. 

b.  Transmit  the  950-Hz  pilot  tone  through  the  test  link  and  record  the 
clear-channel  AI. 

c.  Repeat  the  procedures  and  record  the  AI  scores  under  the  conditions 
stated  by  the  test  plan. 


5.3  Test  Notes 


Data  to  be  recorded  in  addition  to  specific  instructions  listed  in  each 
subtest  shall  include: 

a.  A  block  diagram  of  the  test  setup  employed  in  each  specified  test. 
The  block  diagram  shall  identify,  by  model  and  serial  number,  all  test 
equipment  and  interconnections  (cable  lengths,  connectors,  attenuators,  etc.) 
and  indicate  control  and  dial  settings  where  necessary. 

b.  An  engineering  logbook  containing,  in  chronological  order,  pertinent 
remarks  and  observations  which  would  aid  in  a  subsequent  analysis  of  the  test 
data . 


c.  A  description  of  any  special  voice  processing  circuitry  employed  by 
the  test  item  and  an  analysis  of  their  effect  on  test  results. 
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5 .4  Test  Data 

The  following  data  shall  be  recorded  as  required  by  the  test  plan: 

a.  Sensitivity. 

b.  Signal-to-interference  ratio. 

c.  Signal-to-noise  ratio. 

d.  Adjacent  channel  frequencies. 

e.  Crew  AS. 

f.  Unbiased  standard  deviation  of  the  AS  values. 

g.  Separate  AS  values  for  male  versus  female  speakers. 

h.  Frequently  missed  words. 

i.  Individual  listener  scores  for  each  speaker. 

j.  Word  lists  employed. 

k.  AI. 

6.  DATA  REDUCTION  AND  PRESENTATION 

Processing  of  raw  test  data  shall,  in  general,  consist  of  organizing, 
marking  for  correlation  and  identification,  and  grouping  of  the  test  data. 
Test  item  specifications  shall  be  noted  on  the  test  data  presentation  to 
facilitate  analysis  and  comparison.  Comparisons  can  be  made  with  scores  ob¬ 
tained  on  other  systems  where  identical  tests  were  conducted.  The  data  shall 
be  reduced  and  presented  in  accordance  with  the  test  plan. 


Recommended  changes  to  this  publication  should  be  forwarded  to  Commander, 
US  Army  Test  and  Evaluation  Command,  ATTN:  DRSTE-AD-M,  Aberdeen  Proving 
Ground,  MD,  21005.  Technical  information  may  be  obtained  from  the  preparing 
activity:  Commander,  US  Army  Electronic  Proving  Ground,  ATTN:  STEEP-MT-T, 

Ft  Huachuca,  AZ  85613.  Additional  copies  are  available  from  the  Defense 
Technical  Information  Center,  Cameron  Station,  Alexandria,  VA  22314.  This 
document  is  Identified  by  the  accession  number  (AD  No.)  printed  on  the  first 
page. 
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APPENDIX  A 

DESCRIPTION  OF  THE  VOICE  SCORING  FACILITY 

The  Voice  Scoring  Facility  consists  of  a  specially  constructed,  acoustic¬ 
ally  insulated  listener  and  equipment  room,  and  the  following  instrumentation: 
high  fidelity  audio  magnetic  tape  recorder/reproducers,  high  fidelity  head¬ 
sets,  high  fidelity  microphone,  a  microprocessor  control  unit,  cathode  ray 
tube  (CRT)  display  consoles  with  selector  switches,  and  punched  tape  equip¬ 
ment  . 

The  physical  arrangement  of  the  listener  and  equipment  room  is  shown  in 
figure  A-l .  Provisions  are  made  for  eight  listener  consoles  and  one  test  con¬ 
troller.  The  test  controller  may  view  and  talk  to  the  listeners  and  monitor 
sounds  occurring  in  the  listener  room.  A  functional  block  diagram  is  shown  in 
figure  A-2. 

Testing  is  conducted  under  a  consistent  and  strict  experimental  design. 

In  operation,  tape  recorded  material  to  be  scored  is  presented  to  the  listen¬ 
ers,  who  register  their  responses  by  pressing  the  appropriate  pushbutton  on 
their  respective  display  consoles.  (Table  A-l  shows  samples  of  the  PB  word 
lists  used  in  the  test  tapes.) 

CRT  screens  are  used  to  display  the  preselected  word  groupings  to  the 
listeners.  The  displays  consist  of  key  words  used  in  test  carrier  phrases. 
These  word  groupings  appear  in  a  five  line  format  with  each  line  containing  10 
words.  Each  word  is  positioned  directly  above  a  selector  pushbutton  switch. 
The  listeners  press  the  button  under  the  key  word  (as  they  understand  it)  each 
time  a  test  sentence  is  heard.  A  given  display  contains  the  words  from  two 
word  lists  to  reduce  the  possibility  of  the  listener  using  a  process  of  elimi¬ 
nation  in  masking  devices.  Also,  the  design  of  the  instrumentation  facili¬ 
tates  a  force-choice  mode  of  operation  by  holding  up  the  next  word  until  each 
listener  has  responded,  even  if  only  by  guessing. 

After  the  listener  has  pushed  a  switch  to  indicate  his  word  choice,  the 
selector  buttons  are  disabled  to  prevent  any  change,  and  the  control  unit 
codes  and  retains  the  word  choices.  A  new  test  sentence  is  presented  to  the 
listeners,  and  the  cycle  is  repeated  until  all  50  words  have  been  scored. 

Upon  completion  of  the  word  list  test,  the  control  unit  calculates  AS  and 
standard  deviation  and  punches  a  paper  tape  which  records  all  listener  re¬ 
sponses  to  each  word  within  the  list.  This  paper  tape  may  be  used  for  further 
reduction  of  the  data  as  required. 
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TABLE  A-I.  REPRESENTATIVE  PHONETICALLY  BALANCED  (PB)  WORD  LISTS 


PB-50  List  9 

PB-50  List 

10 

1. 

arch 

26. 

with 

1. 

goose 

26. 

ail 

2. 

year 

27. 

noose 

2. 

slug 

27. 

youth 

3. 

fluff 

28. 

fume 

3. 

rape 

28. 

etch 

4. 

phone 

29. 

rude 

4. 

nudge 

29. 

put 

5. 

crowd 

30. 

hoof 

5. 

page 

30. 

cue 

6. 

lit 

31. 

foe 

6. 

rip 

31. 

line 

7. 

gate 

32. 

club 

7. 

hat 

32. 

force 

8. 

pact 

33. 

fuse 

8. 

valve 

33. 

plus 

9. 

thank 

34. 

mass 

9. 

fir 

34. 

tag 

10. 

chest 

35. 

sip 

10. 

cow 

35. 

clothe 

11. 

weak 

36. 

ten 

11. 

flight 

36. 

wake 

12. 

throne 

37. 

ditch 

12. 

maze 

37. 

cord 

13. 

than 

38. 

give 

13. 

those 

38. 

scrub 

14. 

reed 

39. 

spud 

14. 

staff 

39. 

void 

15. 

birth 

40. 

toad 

15. 

rush 

40. 

mope 

16. 

itch 

41. 

troop 

16. 

snipe 

41. 

champ 

17. 

boost 

42. 

beef 

17. 

ears 

42. 

lap 

18. 

carve 

43. 

cud 

18. 

gull 

43. 

flaunt 

19. 

key 

44. 

root 

19. 

earth 

44. 

pink 

20. 

nuts 

45. 

bit 

20. 

bash 

45. 

daub 

21. 

odd 

46. 

ice 

21. 

jay 

46. 

real 

22. 

chess 

47. 

grace 

22. 

bug 

47. 

bob 

23. 

wipe 

48. 

clown 

23. 

thug 

48. 

chance 

24. 

flag 

49. 

smart 

24. 

tree 

49. 

wade 

25. 

wild 

50. 

nerve 

25. 

back 

50. 

hurt 
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APPENDIX  B 

DESCRIPTION  OF  THE  VOICE  INTELLIGIBILITY  ANALYSIS  SYSTEM 

The  articulation  index  (AI)  value  of  a  voice  communication  link  may  be 
determined  under  field  or  laboratory  conditions  by  the  Voice  Intelligibility 
Analysis  System  (VIAS).  The  VIAS  equipment  consists  of  voice  analyzer  sets, 
self-check  signal  generators,  and  test-signal  generators,  with  their  associ¬ 
ated  amplifiers  and  power  supplies.  The  test-signal  generator  supplies  a 
950-Hz  tone,  which  is  used  operationally  as  the  modulation  signal  for  the  link 
under  test.  To  account  for  the  effect  of  speech  clipping,  which  is  employed 
on  many  military  transmitters,  the  950-Hz  tone  is  modulated  with  a  5-Hz 
signal.  The  pilot  tone,  which  has  the  same  rms  value  as  the  long  term  value 
of  English  speech,  is  transmittd  over  the  test  link  instead  of  speech.  The 
voice  analyzer  set,  or  basic  VIAS,  performs  the  AI  determination  on  the  basis 
of  received  signal. 

The  VIAS  accounts  for  amplitude  distortion,  intermittent  noise,  and  up¬ 
ward  spreading  of  masking.  The  AI  depends  on  the  property  of  speech  intelli¬ 
gibility  which  allows  the  speech  spectrum  to  be  divided  into  several  frequency 
bands,  each  band  contributing  equally  to  the  intelligibility  of  the  total 
speech. 

In  essence,  the  AI  is  determined  by  measuring  the  signal-to-noise  ratio 
in  each  of  the  bands  logarithmically  and  then  summing  over  the  bands.  Smooth¬ 
ing  is  provided  by  integrating  the  summed  results  over  a  period  of  17  seconds. 
Contribution  switches  for  the  individual  bands  are  preset  to  ZERO,  HALF,  or 
FULL,  to  compensate  for  the  predetermined  audio  pass  band  characteristics 
peculiar  to  each  test  item  type  or  configuration.  The  concept  of  testing  by 
means  of  the  VIAS  is  depicted  in  figure  B-l . 
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,  APPENDIX  C 

SUBJECTIVE  VOICE  TRANSMISSION  QUALITY  TEST 


1.  SCOPE.  This  test  procedure  outlines  a  test  method  for  use  in  performing 
subjective  evaluations  of  voice  transmission  quality  over  voice  communications 
equipment.  It  defines  voice  transmission  quality  in  terms  of  the  signal 
strength  (volume),  degrading  effects  of  noise,  and  the  overall  quality  of  the 
voice  channel  under  test.  Each  of  these  three  elements  is  rated  subjectively 
on  a  scale  of  1  (one)  to  5  (five)  based  on  the  definitions  listed  at  the  top 
of  page  C-3.  The  scope  of  this  procedure  is  limited  to  test  programs  that  do 
not  require  quantitative  evaluations  of  voice  performance.  This  procedure 
should  not  be  used  where  the  effectiveness  (intelligibility)  of  the  voice 
communications  is  a  critical  issue.  The  requirements  should  be  identified  in 
the  Independent  Evaluation  Plan  or  other  requirements  documents. 

2.  FACILITIES  AND  INSTRUMENTATION.  No  special  facilities  or  instrumentation 
are  required  other  than  what  is  required  for  the  normal  operation  and  mainte¬ 
nance  of  the  test  items.  Refer  to  the  instruction  manuals. 

3.  PREPARATION  FOR  TESTS. 

a.  Screen  test  operators/collectors  for  hearing  impairment.  Conduct 
hearing  tests  by  appropriate  medical  personnel  if  there  is  any  question  of 
hearing  capabilties. 

b.  Brief  participating  test  personnel  on  all  aspects  of  the  test  program 
to  include  the  purpose  of  the  test  and  the  requirements  for  precision  during 
test  conduct. 

c.  Provide  sufficient  copies  of  operating  instructions/detailed  test 
procedures  to  all  participating  test  personnel. 

d.  Perform  the  operational  checks,  maintenance  procedures,  alignment,  and 
calibration  steps  specified  in  pertinent  instruction  manuals  to  insure  that 
the  test  item(s)  is  in  proper  operating  condition.  Prepare  the  test  item(s) 
insofar  as  possible,  to  represent  the  equipment  in  normal  operating  condition. 

4.  TEST  CONTROLS.  Consider  acoustical  noise  in  the  placement  of  the  test 
item(s)  and  during  the  conduct  of  the  test.  Avoid  high  ambient  noise  sources, 
such  as  power  generating  equipment  and  aircraft,  if  they  are  not  representa¬ 
tive  of  the  tactical  operational  environment. 

5.  PERFORMANCE  TESTS. 

a.  Verify  the  placement  and  setup  of  the  test  item(s)  and  support  equip¬ 
ment  . 

b.  Power  up  the  equipment  and  conduct  appropriate  performance  checks  as 
listed  in  the  instruction  manuals. 


C-l 


19  November  1981 


TOP  6-2-521 


c.  Verify  voice  communications  on  the  selected  channel. 

d.  Complete  the  data  collection  sheet.  Describe  fully  in  the  remarks 
section  of  the  data  sheet  any  adverse  conditions  such  as  distortions,  back¬ 
ground  noise,  speech  clarity,  etc. 

e.  Repeat  steps  c  and  d  for  the  opposite  direction  of  transmission. 

f.  Repeat  steps  c  through  e  for  each  of  the  selected  test  channels  as 
indicated  in  the  detailed  test  procedure. 


6.  DATA  REDUCTION  AND  PRESENTATION.  Review  the  data  collection  sheets  and 
present  the  data  in  tabular  form.  Include  other  pertinent  performance  data  in 
the  tabular  data. 

DATA  COLLECTION  SHEET 

1. 

Test  Item(s):  2.  SN: 

3. 

Test  Sequence  No.:  4.  Test  Operator: 

5. 

Date:  6.  Time: 

7. 

Detailed  Test  Procedure  Para  No.  (Step  No.): 

8. 

Channel  ID  (No.): 

9. 

Calling  Party  ID  (No.): 

10. 

Called  Party  ID  (No.): 

11. 

Subjective  Evaluation  (Circle  appropriate  number) 

11a. 

Signal  Strength  (volume)  lib.  Noise  Level  11c.  Readability  (Quality) 

5 

Excellent 

5 

Nil 

5 

Excellent 

4 

Good 

4 

Slight 

4 

Good 

3 

Fair 

3 

Moderate 

3 

Fair 

2 

Poor 

2 

Severe 

2 

Poor 

1 

Barely  Audible' 

1 

Extreme 

1 

Unusable 

12.  Remarks:  (describe  fully  any  adverse  conditions): 
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DEFINITIONS  OF  RATING  ELEMENTS* 


Rating  Scale 


Rating  Element 


Condition 


5 


Excellent 


Signal  (voice)  quality  unaffected 
by  the  transmission  media. 


4 


Good 


Signal  (voice)  quality  slightly 
affected . 


3  Fair  Signal  (voice)  quality  seriously 

affected.  Channel  usable  by 
operators  but  not  suitable  for 
mission  traffic. 


2  Poor  Channel  just  usable  by  operators. 

1  Unusable  Channel  unusable  by  operators. 

TEST  OPERATIONS  CHECKLIST 

1.  Test  item(s)  and  support  equipment  in  place  and  in  proper 

operating  condition  _ 


2.  Operator  personnel  have  been  briefed  on  conduct  of  test 

3.  Operating  instruct ions /detailed  test  procedures  and  data 
collection  sheets  have  been  provided  to  all  test  operators/ 
data  collectors 


4.  Perform  test  _ 

5.  Review  test  data  _ _ 

6.  Retest,  if  required  _ 

7.  Prepare  test  data  in  final  reduced  format  _ 

REFERENCES 

1.  "Reference  Data  for  Radio  Engineers",  Howard  W.  Sams  &  Co.,  Inc.,  Fifth 
Edition,  October  1968. 

2.  "Transmission  Systems  for  Communications",  Bell  Telephone  Laboratories, 
Inc.,  Revised  Fourth  Edition,  December  1971. 


*From  ITU  Radio  Regulations,  Geneva,  1959 
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